Image processing apparatus, image processing method, and image processing program

ABSTRACT

There is provided an image processing apparatus for applying an adjusting process to an image including multicolor image signals. An image area including a pixel to be processed is extracted, and one representative signal value is generated from signal values corresponding to a plurality of colors of pixels included in the image area (S 2501 ). A feature amount associated with the image area is calculated based on the representative signal value (S 2503 , S 2505 , S 2511 ), and a substitute candidate pixel common to the plurality of colors is selected from the image area based on the calculated feature amount (S 2512 ). Signal values of the pixel to be processed are substituted by new pixel values, which are calculated from the signal values of the pixel to be processed and signal values of the substitute candidate pixel.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an image processing apparatus forperforming an image adjusting process, image processing method, andimage processing program thereof.

2. Description of the Related Art

Conventionally, an image processing apparatus which applies an adjustingprocess to an image is known. For example, the edge of an imageoptically scanned by a scanner of a copying machine is more blurred thanthat of the original document image, and an image with poor sharpness isobtained when it is printed on a print medium intact. Upon applying anedge emphasis filter as an edge emphasis process after scanning for thepurpose of enhancing the sharpness, an unnatural image is formed sincethe character edge is emphasized too strongly due to so-called overshootor undershoot. Also, with this process, noise and moiré generated uponscanning are emphasized. In order to solve these problems, a techniquethat substitutes the pixel of interest by surrounding pixels is proposed(Japanese Patent No. 2620368).

However, with the aforementioned prior art, if there are multicolorimage signals, appropriate adjusting means cannot be provided. Forexample, with the technique described in Japanese Patent No. 2620368, ifa discrimination process is executed based on the distribution patternof image signals independently for each of a plurality of colors, imageemphasis signals suited to a color image cannot be generated, resultingin color misregistration.

Furthermore, upon checking the distribution pattern of image signal, ifa large filter size is used, the calculation volume increases, resultingin an increase in volume of a process for generating an poorly definedsignal.

SUMMARY OF THE INVENTION

There is provided an apparatus, method and program for performing anappropriate image-adjusting process to multicolor images.

According to one aspect of the present invention, the foregoing problemis solved by providing an image processing apparatus for applying anadjusting process to an image including multicolor image signals,comprising: an extracting unit arranged to extract an image areaincluding a pixel to be processed; a generation unit arranged togenerate one representative signal value from signal valuescorresponding to a plurality of colors of pixels included in the imagearea; a feature amount calculation unit arranged to calculate a featureamount associated with the pixel to be processed based on therepresentative signal value; pixel common to the plurality of colorsfrom the image area based on the calculated feature amount; and a unitarranged to substitute signal values of the pixel to be processed by newpixel values, which are calculated from the signal values of the pixelto be processed and signal values of the substitute candidate pixel.

According to another aspect of the present invention, the foregoingproblem is solved by providing an image processing apparatus forapplying an adjusting process to an image including multicolor imagesignals, comprising: an extracting unit arranged to extract an imagearea including a pixel to be processed; an edge direction determinationunit arranged to determine an edge direction in the image area; aselection unit arranged to select a substitute candidate pixel from apixel group included in the edge direction; and a unit arranged tosubstitute signal values of the pixel to be processed by new pixelvalues, which are calculated from the signal values of the pixel to beprocessed and signal values of the substitute candidate pixel.

According to still another aspect of the present invention, theforegoing problem is solved by providing an image processing apparatusfor applying an adjusting process to an image including multicolor imagesignals, comprising: an extracting unit arranged to extract an imagearea including a pixel to be processed; a generation unit arranged togenerate one representative signal value from signal valuescorresponding to a plurality of colors of pixels included in the imagearea; a feature amount calculation unit arranged to calculate a featureamount associated with the pixel to be processed based on therepresentative signal value; a unit arranged to select a substitutecandidate pixel common to the plurality of colors from the image areabased on the calculated feature amount; an edge direction determinationunit arranged to determine an edge direction based on the representativesignal value in the image area; a selection unit arranged to select asubstitute candidate pixel from a pixel group included in the edgedirection; and a unit arranged to substitute signal values of the pixelto be processed by new pixel values, which are calculated from thesignal values of the pixel to be processed and signal values of thesubstitute candidate pixel.

According to yet another aspect of the present invention, the foregoingproblem is solved by providing an image processing method for applyingan adjusting process to an image including multicolor image signals,comprising the steps of: extracting an image area including a pixel tobe processed; generating one representative signal value from signalvalues corresponding to a plurality of colors of pixels included in theimage area; calculating a feature amount associated with the pixel to beprocessed based on the representative signal value; selecting asubstitute candidate pixel common to the plurality of colors from theimage area based on the calculated feature amount; and substitutingsignal values of the pixel to be processed by new pixel values, whichare calculated from the signal values of the pixel to be processed andsignal values of the substitute candidate pixel.

According to still yet another aspect of the present invention, theforegoing problem is solved by providing an image processing method forapplying an adjusting process to an image including multicolor imagesignals, comprising the steps of: extracting an image area including apixel to be processed; generating one representative signal value fromsignal values corresponding to a plurality of colors of pixels includedin the image area; calculating a feature amount associated with thepixel to be processed based on the representative signal value;selecting a substitute candidate pixel common to the plurality of colorsfrom the image area based on the calculated feature amount; determiningan edge direction based on the representative signal value in the imagearea; selecting a substitute candidate pixel from a pixel group includedin the edge direction; and substituting signal values of the pixel to beprocessed by new pixel values, which are calculated from the signalvalues of the pixel to be processed and signal values of the substitutecandidate pixel.

According to yet still another aspect of the present invention, theforegoing problem is solved by providing an image processing program forapplying an adjusting process to an image including multicolor imagesignals, the program making a computer execute the steps of: extractingan image area including a pixel to be processed; generating onerepresentative signal value from signal values corresponding to aplurality of colors of pixels included in the image area; calculating afeature amount associated with the pixel to be processed based on therepresentative signal value; selecting a substitute candidate pixelcommon to the plurality of colors from the image area based on thecalculated feature amount; and substituting signal values of the pixelto be processed by new pixel values, which are calculated from thesignal values of the pixel to be processed and signal values of thesubstitute candidate pixel.

According to still yet another aspect of the present invention, theforegoing problem is solved by providing an image processing program forapplying an adjusting process to an image including multicolor imagesignals, the program making a computer execute the steps of: extractingan image area including a pixel to be processed; generating onerepresentative signal value from signal values corresponding to aplurality of colors of pixels included in the image area; calculating afeature amount associated with the pixel to be processed based on therepresentative signal value; selecting a substitute candidate pixelcommon to the plurality of colors from the image area based on thecalculated feature amount; determining an edge direction based on therepresentative signal value in the image area; selecting a substitutecandidate pixel from a pixel group included in the edge direction; andsubstituting signal values of the pixel to be processed by new pixelvalues, which are calculated from the signal values of the pixel to beprocessed and signal values of the substitute candidate pixel.

Further features of the present invention will become apparent from thefollowing description of exemplary embodiments with reference to theattached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIGS. 1A and 1B are explanatory views of an MFP;

FIG. 2 is a block diagram for explaining the control arrangement of theMFP;

FIG. 3 is a flowchart of an image process of the MFP;

FIGS. 4A to 4C are explanatory views of a process unit;

FIG. 5 is a flowchart of movement of a process unit;

FIGS. 6A and 6B are explanatory views of the scan range of an imagesensing element;

FIGS. 7A to 7F are graphs for explaining the luminance, and primaryderivative, and secondary derivative;

FIG. 8 is an explanatory view of 4-direction extraction;

FIG. 9 is an explanatory view of an L difference;

FIGS. 10A to 10D are graphs for explaining a variation time count;

FIGS. 11A to 11C are graphs for explaining a variation time countadjusting process;

FIGS. 12A to 12D are graphs for explaining an edge level definingprocess;

FIGS. 13A to 13C are graphs for explaining an edge emphasis process;

FIG. 14 is a flowchart of an adjusting level defining process accordingto the embodiment;

FIGS. 15A to 15C are graphs for explaining a substitute level definingprocess;

FIG. 16 is a flowchart of an adjusting process according to theembodiment;

FIG. 17 is a table for explaining an image with poor sharpness of anedge as a result of scanning;

FIG. 18 is a table showing the execution result of the embodiment;

FIG. 19 is a table showing an example of the edge emphasis resultwithout using a representative value signal;

FIG. 20 is an explanatory view of four directions when a reference areais changed; and

FIGS. 21A and 21B are explanatory views of edge emphasis filtercoefficients.

DESCRIPTION OF THE EMBODIMENTS

A preferred embodiment of the present invention will now be described indetail with reference to the drawings. It should be noted that therelative arrangement of the components, the numerical expressions andnumerical values set forth in these embodiments do not limit the scopeof the present invention unless it is specifically stated otherwise.

<Arrangement of MFP>

FIGS. 1A and 1B are schematic perspective views of a multi functionprinter (to be abbreviated as MFP hereinafter) 1 according to oneembodiment of the present invention. FIG. 1A shows a state in which anADF (auto document feeder) 31 of the MFP 1 is closed, and FIG. 1B showsa state in which the ADF 31 is open.

This MFP 1 basically has a function of printing data received from ahost computer (PC) as a printer, and a function as a scanner.Furthermore, functions of the MFP alone include a copy function ofprinting an image scanned by the scanner using the printer, a functionof printing image data read from a storage medium such as a memory cardor the like, and a function of printing image data received from adigital camera or the like.

Referring to FIGS. 1A and 1B, the MFP 1 comprises a scanning unit 14such as a flatbed scanner or the like, and a print unit 13 of an ink-jetsystem, electrophotography system, or the like. Also, the MFP 1comprises a display unit 19 such as a display panel or the like, and anoperation unit 15 including various key switches, and the like. The MFP1 comprises a USB port (not shown) used to communicate with the PC onits back surface of the MFP 1, and the USB port allows the MFP 1 tocommunicate with the PC. The MFP 1 comprises a card interface 22including a card slot used to read out data from various memory cards,and a camera interface 23 including a camera port used to make datacommunications with a digital camera. In addition, the MFP 1 comprisesthe ADF 31 used to automatically set a document on a document table.

FIG. 2 is a block diagram showing the internal arrangement of the MFP 1.Referring to FIG. 2, a CPU 11 controls various functions of the MFP 1,and executes image process programs stored in a ROM 16 in accordancewith predetermined operations at the operation unit 15. By executing theprograms, the CPU 11 can select a pixel to be processed, and can extractan image area with a predetermined size including the pixel to beprocessed. The CPU 11 can calculate a variation, variation time count,and variation acceleration associated with a pixel to be processed basedon signal values of pixels included in the image area. The ROM 16 storesvarious tables and mathematical formulas used in image processes, andserves as a defining unit which defines correspondence among thevariation, variation time count, and variation acceleration, and variousadjusting levels. The scanning unit 14 which comprises a CCD scans adocument image, and outputs red (R), green (G), and blue (B) analogluminance data. Note that the scanning unit 14 may comprise a contactimage sensor (CIS) in place of the CCD. Also, when the MFP 1 comprisesthe ADF 31, it is convenient to continuously scan order sheets.

The card interface 22 loads image data, which is captured by, e.g., adigital still camera (to be abbreviated as DSC hereinafter) and isrecorded on a memory card or the like, in accordance with operations atthe operation unit 15. Note that the color space of the image dataloaded via the card interface 22 is converted from that (e.g., YCbCr) ofthe DSC into a standard RGB color space (e.g., NTSC-RGB or sRGB) ifnecessary. The loaded image data undergoes various kinds of processesrequired for an application such as resolution conversion to theeffective number of pixels, and the like, based on its headerinformation. The camera interface 23 is used to directly connect the DSCand to read image data.

An image processor 12 executes image processes such as conversion of aread signal value, an image adjusting/modification process, conversionfrom luminance signals (RGB) into density signals (CMYK), scaling, gammaconversion, error diffusion, and the like. The adjusting process to beexecuted by the image processor 12 includes an edge emphasis process,smoothing process, substitution process, achromatization process, andthe like, and the image processor 12 serves as an adjusting unit. Dataobtained by the image processes in the image processor 12 is stored in aRAM 17. When adjusted data stored in the RAM 17 reaches a predeterminedamount, the print unit 13 executes a print operation.

A nonvolatile RAM 18 comprises, e.g., a battery backed-up SRAM or thelike, and stores data unique to the MFP 1 or the like. The operationunit 15 comprises a photo direct print start key which allows the userto select image data stored in a storage medium (memory card) and tostart printing. The operation unit 15 also comprises a key used to printan order sheet, a key used to scan an order sheet, and the like. Theoperation unit 15 may also comprise a copy start key in a monochromecopy mode or color copy mode, a mode key used to designate a mode suchas a copy resolution, image quality, and the like, a stop key used tostop a copy operation or the like, a numerical keypad used to input acopy count, a registration key, and the like. The CPU 11 detects thepressing states of these keys and controls respective units according tothe detected states.

The display unit 19 comprises a dot matrix type liquid crystal displayunit (LCD) and an LCD driver, and makes various displays under thecontrol of the CPU 11. Also, the display unit 19 displays thumbnails ofimage data recorded in a storage medium. The print unit 13 comprises anink-jet head of an ink-jet system, general-purpose IC, and the like. Theprint unit 13 reads out print data stored in the RAM 17 and prints itout as a hard copy under the control of the CPU 11.

A drive unit 21 includes stepping motors for driving feed and dischargerollers, gears for transmitting the driving forces of the steppingmotors, a driver circuit for controlling the stepping motors, and thelike in order to operate the scanning unit 14 and print unit 15.

A sensor unit 20 includes a print sheet width sensor, print sheetsensor, document width sensor, document sensor, print medium sensor, andthe like. The CPU 11 detects the states of a document and print sheetbased on information obtained from this sensor unit 20.

A PC interface 24 is an interface with the PC, and the MFP 1 performs aprint operation, scan operation, and the like from the PC via the PCinterface 24. In a copy operation, image data scanned by the scanningunit 14 undergoes a data process inside the MFP, and is printed usingthe print unit 13.

Upon instruction of a copy operation from the operation unit 15, thescanning unit 14 scans a document set on the document table. The scanneddata is sent to the image processor 12 and undergoes the image process.Then, the processed data is sent to the print unit 13, thus executing aprint process.

<Image Process>

FIG. 3 is a flowchart of the image process executed in a copy mode.Respective steps will be described below. The CPU 11 applies shadingcorrection to image data, which is scanned by the scanning unit 14 andis A/D-converted, so as to adjust variations of an image sensing elementin STEP 301.

After that, the CPU 11 executes input device color conversion in STEP302. As a result, the device-dependent color space of signal data isconverted into a standard color space domain. For example, the standardcolor space includes sRGB specified by IEC (InternationalElectrotechnical Commission). Also, AdobeRGB propounded by Adobe SystemsIncorporated may be used. The conversion method includes an arithmeticmethod using a 3×3 or 3×9 matrix, a lookup table method which determinesvalues based on a table that describes conversion rules, and the like.

In STEP 303, the CPU 11 applies an adjusting/modification process to theconverted data. The process contents include an edge emphasis processthat adjusts blurring due to scanning, a text modification process thatimproves legibility of text, a process for removing bleed-through thathas occurred due to scanning upon light irradiation, and the like.

In STEP 304, the CPU 11 executes an enlargement/reduction process toconvert the data to a desired scale when the user designates a zoomscale, in a 2-in-1 copy mode that assigns two document images on onesheet, or the like. As the conversion method, methods such as bicubic,nearest neighbor, and the like are generally used.

In STEP 305, the CPU 11 converts the data on the standard color spaceinto signal data unique to an output device. The MFP according to thisembodiment adopts an ink-jet system, and executes a conversion processinto ink color data such as cyan, magenta, yellow, black, and the like.This conversion can use the same method as in STEP 302.

Furthermore, in STEP 306 the CPU 11 converts the data into the number ofprintable levels. For example, in case of binary expression, i.e.,ON/OFF of ink dots, the data may be binarized by a quantization methodsuch as error diffusion or the like. As a result, the data is convertedinto a data format that the printer can print, and a print operation isexecuted based on that data, thus forming an image.

<Process Unit>

FIG. 4A is a view for explaining a process unit upon execution of anattribute determination process. Assuming that a pixel indicated by“circle” in FIG. 4A is a pixel of interest (pixel to be processed), anarea defined by 7×7 pixels including the pixel of interest (7×7 area) isdefined, as indicated by the bold line in FIG. 4A. An image process forthe pixel of interest is executed using image signals in this defined7×7 area. After execution of the process of the pixel of interest, apixel that neighbors the pixel of interest is defined as the next pixelof interest, as indicated by “X” in, e.g., FIG. 4B, and a 7×7 area issimilarly defined to execute the image process. After that, the pixel ofinterest is similarly shifted pixel by pixel in turn and a new 7×7 areais defined in each case, thus adjusting all pixels to be processed.

A case will be explained below wherein the process unit is an area unit.A 7×7 area is defined for a pixel indicated by “circle” in FIG. 4A, andan adjusting level to be defined for “circle” is applied to a pluralityof pixels, e.g., all pixels, in the 7×7 area. As the next process unit,a 7×7 area is defined for a pixel indicated by “triangle” in FIG. 4C. Inthis way, the process unit is shifted, so that the 7×7 area for “circle”neighbors that for “triangle”.

However, since the pixel unit is preferably used as the process unit todefine the adjusting level at higher accuracy, the pixel unit will bedescribed as the process unit in this embodiment.

FIG. 5 is a flowchart showing a shift procedure of the process unit. InSTEP 501, the CPU 11 defines a process target (pixel to be processed).Immediately after START, the CPU 11 defines a first pixel to beprocessed. If the process returns from STEP 505 to STEP 501, the CPU 11defines the next pixel to be processed.

STEP 502 is a process area defining step. The process area is an areaconfigured by a plurality of pixels (7×7 area in the above description)including the process unit, as described above.

STEP 503 is an adjusting level defining step. The CPU 11 sets anadjusting level for the process unit.

STEP 504 is an adjusting execution step. The CPU 11 adjusts the processunit using the adjusting level defined in STEP 503.

STEP 505 is a last adjusting target checking step. That is, the CPU 11checks if the process unit is the last one. If the process unit is notthe last one (NO), the process returns to STEP 501. If the process unitis the last one (YES), the process reaches “END”.

In the embodiments to be described hereinafter, the 7×7 area is used asthe process area. This is because the pixel range of a document to bescanned by one pixel of the image sensing element (CCD or CIS) used inthe scanning unit is designed to include six pixels or less. Althoughthe pixel range to be scanned is designed to include six pixels or less,reflected light from a document that enters the image sensing elementreceives various influences due to float of a document from the documenttable, unevenness of a document, and the like. For this reason, onepixel of the image sensing element may scan a range exceeding sixpixels. The embodiments to be described hereinafter show a plurality ofgraphs used to explain image signals obtained by scanning a document.However, these image signals are not always obtained from reflectedlight within six pixels or less.

FIGS. 6A and 6B simply illustrate the range of reflected light thatcomes from a document and enters one pixel of the image sensing element.As shown in FIG. 6A, the image sensing element used in this embodimentis designed so that reflected light within six pixels from a 7-pixelrange of a document enters one pixel of the image sensing element (asdescribed above, reflected light that exceeds six pixels may enter insome cases). That is, reflected light from one pixel on a documentinfluences seven pixels of the image sensing element. This causes edgeblurring described in the paragraphs of the related art, anddeteriorates sharpness.

The embodiment to be described hereinafter aims at reducing blurring asone of its objects. An edge is emphasized by substituting the pixel ofinterest by a substitute candidate pixel. Hence, by selecting asubstitute candidate from a pixel area less influenced by a documentpixel corresponding to the pixel of interest, the edge emphasis effectcan be enhanced. For this reason, an area which is influenced by onepixel of a document image is assured as the process area. Therefore, the7×7 area is defined as the process area. In order to enhance the edgeemphasis effect more, it is effective to set an area exceeding 7×7pixels as a reference area. Also, when one pixel of the image sensingelement is designed to receive reflected light from a 3-pixel range, asshown in FIG. 6B, the process area may be defined as small as a 3×3area.

In order to reduce the process data size for the purpose of speeding upprocesses, an image is often scanned at a resolution lower than that ofthe image sensing element by decimating valid pixels, summing andintegrating neighboring pixels, or the like. In such a case, it iseffective to define an area smaller than the 7×7 area as a referencearea. Conversely, when an image is scanned at a higher resolution, it iseffective to define a process area to be larger than the 7×7 area.

In this manner, the reference area can be defined as needed inaccordance with the performance of the image sensing element such as thenumber of pixels of the image sensing element influenced by one pixel ofa document image, the spot size, the number of blurred pixels,Modulation Transfer Function (MTF), and the like.

<Definition of Terms>

Terms used in the present specification will be defined below.

A variation is a value that represents the magnitude of the variationsof pixel signal values in a surrounding pixel group having a pixel to beprocessed as the center. In this embodiment, of the absolute values(edge amounts) of differences between luminance values of two pixelsthat neighbor one pixel on two sides, a maximum one will be explained asthe variation. However, the present invention is not limited to such aspecific value. For example, the variation may be a value that expressesthe difference (magnitude) of changes such as the absolute value of theprimary derivative of a value related to an image signal of a pixel ofinterest or the like, or a value that representatively expresses thedifference (magnitude) of changes of values related to image signals inan area of interest.

A variation time count is a value that represents the frequency ofoccurrence of variations of image signal values in a surrounding pixelgroup having the pixel to be processed as the center. In thisembodiment, the frequency of occurrence of increment/decrement of thedifference between the luminance values of two pixels that neighbor onepixel on two sides in the image area (the number of sign changes (thenumber of zero-crossing points)) will be described as the variation timecount. However, the present invention is not limited to such specificvalue.

The variation time count is defined as a value that expresses thefrequency of occurrence of changes in value associated with imagesignals such as the number of zero-crossing points or spatial frequencyof the primary derivatives of values associated with image signals inthe image area, a black-white change count after binarization, and thelike.

A variation acceleration is a value that represents the acceleration ofvariations of pixel signal values in a surrounding pixel group havingthe pixel to be processed as the center. In the following embodiment,the variation acceleration will be explained as a value obtained byfurther calculating a difference from the differences of luminancevalues in the image area. However, the present invention is not limitedto such specific value. For example, the acceleration variation may be avalue that expresses the acceleration of changes such as the secondaryderivative of values associated with image signals in an area ofinterest and the like.

A saturation will be explained as a maximum absolute value of imagesignal differences of respective colors of a pixel or area of interestin the following embodiments. However, the present invention is notlimited to such specific value. The saturation is defined as a valuethat expresses the distance from the chromaticity axis.

Adaptively defining an adjusting level is defined such that differentadjusting levels are defined for each value in at least some value areasof those which the defined variation time count, variation, variationacceleration, and saturation can assume.

Details of the image processes of this embodiment will be describedbelow. Note that a range from 0 to 255 that an image signal can assumewill be exemplified. However, the range of the image signal is notlimited to such specific one, and it may be defined to fit in the MFP orimage processes.

FIG. 14 is a flowchart of the adjusting level defining process accordingto this embodiment. The adjusting level defining process will bedescribed below along respective steps of the flowchart.

<Adjusting Level Defining STEP 2501: Define Process Area>

The CPU 11 extracts a process area, i.e., a 7×7 area including sevenpixels in the horizontal direction and seven pixels in the verticaldirection to have the pixel of interest (the pixel to be processed) asthe center in an image configured by RGB multi-valued image signals. TheCPU 11 generates a 7×7 process area of luminance L by calculatingluminance L from respective pixel values R, G, and B of the process areaby:

L=(R+2×G+B)/4  (1)

Note that this embodiment uses luminance L given by equation (1), butmay adapt another luminance. For example, image signals includinglightness and color difference components may be selected, and L* of auniform color space L*a*b* may be used as a luminance. Alternatively, Yof YCbCr may be used as a luminance, or only G of RGB may be extractedand used as L. In this manner, since the arithmetic volume can bereduced by selecting a specific color as the overall representativecolor, circuit implementation can be facilitated.

FIG. 7A shows luminance values upon scanning a black vertical line in awhite background in the horizontal direction. FIG. 7B shows luminancevalues upon scanning halftone dots, that line up in the horizontaldirection in a white background, in the horizontal direction.

<Adjusting Level Defining STEP 2502: Extract Four Directions>

The CPU 11 extracts, from the process area of L generated in STEP 2501,seven pixels in each of a total of four directions, i.e., one horizontaldirection, one vertical direction, and two oblique directions, as shownin FIG. 8.

<Adjusting Level Defining STEP 2503: Calculate L Difference>

The CPU 11 calculates, as feature amounts, differences Grd of L of fivepixels in each direction from L in the four directions extracted in STEP2502, as shown in FIG. 9, and using:

Grd(i)=L(i+1)−L(i−1)  (2)

Where L(i−1) is a pixel before pixel L(i), and L(i+1) is a pixel afterpixel L(i).

Equation (2) represents the primary derivative of digital signal values.Note that the L difference calculation method is not limited to suchspecific method. For example, differences between neighboring pixels maybe calculated, or differences between pixels further separated fromthose before and after a given pixel described above may be calculated.FIGS. 7C and 7D show Grd calculated by applying equation (2) to L inFIGS. 7A and 7B.

<Adjusting Level Defining STEP 2504: Determine Edge Direction)

The CPU 11 calculates Grd absolute values in the four directions of thepixel of interest in Grd in the four directions calculated in STEP 2503.The CPU 11 determines a direction that shows a maximum Grd absolutevalue of those in the four directions as an edge direction of the pixelof interest.

<Adjusting Level Defining STEP 2505: Calculate Variation>

The CPU 11 can calculate Grd values for five pixels in STEP 2503 fromseven pixels that line up in the edge direction determined in STEP 2504.The CPU 11 compares these Grd values for five pixels and calculatestheir maximum absolute value as a variation (edge amount) of the pixelof interest. An edge is stronger with increasing variation, and is closeto flat with decreasing variation.

<Adjusting Level Defining STEP 2506: Calculate Variation Time Count>

The CPU 11 calculates a variation time count as a total of the fourdirections from the Grd values in the four directions calculated in STEP2503. That is, the CPU 11 calculates, as the variation time count (thenumber of zero-crossing points) of the pixel of interest, the number ofchange times of the sign of Grd from + to − or vice versa, as shown inFIG. 10A, or the number of change times of the sign of Grd from + to 0and then to − at the next pixel, or from − to 0 and then to + at thenext pixel, as shown in FIG. 10B.

Note that this embodiment does not count, as the variation time count, acase in which the sign changes to sandwich zeros of a plurality ofpixels, as shown in FIG. 10C, or a case in which the Grd value becomeszero but its sign does not change, as shown in FIG. 10D. The reason forthis will be explained below. When the Grd values assume zero at theplurality of pixels or when the Grd values become zero but their signdoes not change, a bold line is likely to be determined. As will bedescribed in STEP 2508 and STEP 2703, a level different from that for athin line in FIG. 10A or 10B can be defined for the bold line.

As shown in FIGS. 7A to 7F, text tends to have a smaller variation timecount than halftone. However, depending on the density uniformity of adocument or shading accuracy, text often has changes with smallamplitudes in Grd, as shown in FIG. 11A. In such case, the variationtime count increases like halftone, and if an adjusting level definingprocess (to be described later) is executed, an edge level close to thatof halftone is undesirably defined. Hence, when the variation calculatedin STEP 2505 is relatively large, i.e., when it exceeds a definedthreshold, small Grd values are smoothed to zero to define an edge levelwith higher accuracy. More specifically, the variation calculated inSTEP 2505 is compared with a threshold (edge threshold), and if thevariation exceeds the edge threshold, a smoothing threshold is defined,as shown in FIG. 11B. In case of Grd absolute values equal to or smallerthan the smoothing threshold, the variation time count is counted tohave Grd=0, as shown in FIG. 11C. In this manner, the variation timecount of text can be suppressed, and the edge level defining processwith high accuracy can be attained.

<Adjusting Level Defining STEP 2507: Edge Level Defining Process 1 Basedon Variation Time Count>

The CPU 11 adaptively defines an edge level or magnitude Fz1 inaccordance with the variation time count calculated in STEP 2506. FIG.12A is a graph for explaining the Fz1 defining process in STEP 2507: theabscissa plots the variation time count, and the ordinate plots Fz1. Incase of the variation time count which is smaller than a first thresholdand with which a text area is more likely to be determined, Fz1 isdefined to be 1 to emphasize the edge. In case of the variation timecount which is larger than a second threshold and with which a halftonearea that readily causes moiré at high LPI is more likely to bedetermined, Fz1 is defined to be 0 so as not to emphasize moiré. In caseof the variation time count falling within a range between the first andsecond thresholds (both inclusive), Fz1 is adaptively defined toprogressively change according to the variation time count, so thatFz1=1 when the variation time count=the first threshold and Fz1=0 whenthe variation time count=the second threshold. In this way, switching ofthe processes is obscured at the boundary of the edge-emphasized areaand non-emphasized area. More specifically, the CPU 11 can adaptivelydefine Fz1 with reference to FIG. 12A or using:

Fz1=(second threshold−variation time count)/(second threshold−firstthreshold)  (3)

<Adjusting Level Defining STEP 2508: Edge Level Defining Process 2 Basedon Variation Time Count>

The CPU 11 adaptively defines an edge level Fz2 in accordance with thevariation time count calculated in STEP 2506. FIG. 12B is a graph forexplaining the Fz2 defining process in STEP 2508: the abscissa plots thevariation time count, and the ordinate plots Fz2. FIG. 12B aims atattaining FIG. 12C when it is combined with FIG. 12A. As described inSTEP 2506, when the variation time count is zero, a bold line area ismore likely to be determined. When such bold line area undergoes edgeemphasis using an edge emphasis filter (to be described later),bordering in which a border part of the bold line area darkens occurs.In order to prevent bordering, in case of the variation time count whichis smaller than a third threshold and with which the bold line area ismore likely to be determined, as shown in FIG. 12B, Fz2 is defined to be0 to suppress edge emphasis. In case of the variation time count whichis larger than a fourth threshold and with which a thin line area ismore likely to be determined, Fz2 is defined to be 1 to apply edgeemphasis. In case of the variation time count falling within a rangebetween the third and fourth thresholds (both inclusive), Fz2 isadaptively defined to progressively change in accordance with thevariation time count, so that Fz2=0 when the variation time count=thethird threshold and Fz2=1 when the variation time count=the fourththreshold. As a result, switching of the processes can be obscured atthe boundary between the edge-emphasized area and non-emphasized area bythe filter. More specifically, the CPU 11 can adaptively define Fz2 withreference to FIG. 12B or using:

Fz2=(variation time count−third threshold)/(fourth threshold−thirdthreshold)  (4)

Fz1×Fz2 can implement the edge level shown in FIG. 12C. If bordering isrequired, Fz2=1 can be defined irrespective of the variation time count.

<Adjusting Level Defining STEP 2509: Edge Level Defining Process Basedon Variation>

The CPU 11 adaptively defines an edge level Fe in accordance with thevariation calculated in STEP 2505. FIG. 12D is a graph for explainingthe Fe defining process in STEP 2509: the abscissa plots the variation,and the ordinate plots Fe. In case of the variation which is smallerthan a fifth threshold and with which a flat area is more likely to bedetermined, Fe=0 is defined so as not to roughen an image by emphasizingsmall variations. In case of the variation which is smaller than a sixththreshold and with which an edge area is more likely to be determined,Fe=1 is defined to apply edge emphasis. In case of the variation fallingwithin a range between the fifth and sixth thresholds (both inclusive),Fe is adaptively defined to progressively change according to a changein variation, so that Fe=0 when the variation=the fifth threshold andFe=1 when the variation=the sixth threshold. As a result, switching ofthe processes can be obscured at the boundary between theedge-emphasized area and non-emphasized area. More specifically, the CPU11 can adaptively define Fe with reference to FIG. 12D or using:

Fe=(variation−fifth threshold)/(sixth threshold−fifth threshold)  (5)

<Adjusting Level Defining STEP 2510: Determine Maximum and MinimumLuminance Positions>

The CPU 11 determines pixel positions with maximum L and minimum L fromseven pixels of L in the edge direction determined in STEP 2504 of thefour directions extracted in STEP 2502.

<Adjusting Level Defining STEP 2511: Calculate Variation Acceleration>

The CPU 11 calculates a variation acceleration Lap of three pixels fromGrd of the edge direction calculated in STEP 2503 in the edge directiondetermined in STEP 2504 as a feature amount associated with theextracted image area. The CPU 11 calculates the variation accelerationby:

Lap(i)=Grd(i+1)−Grd(i−1)  (15)

where Grd(i−1) is a pixel before pixel Grd(i), and Grd(i+1) is a pixelafter that pixel. FIGS. 7E and 7F respectively show Lap calculated byapplying equation (15) to Grd shown in FIGS. 7C and 7D.

Note that the calculation method of the variation acceleration is notlimited to this. For example, a difference between neighboring Grd datamay be calculated. Also, equation (15) expresses the secondaryderivative of digital signal values.

<Adjusting Level Defining STEP 2512: Determine Substitute PixelPosition>

The CPU 11 determines a substitute pixel position based on the pixelpositions with maximum L and minimum L determined in STEP 2510 and thevariation accelerations Lap calculated in STEP 2511. As shown in FIGS.7A to 7F, when the sign of Lap is +, L of the pixel of interest tends toassume a value, the absolute value of which is closer to minimum L thanmaximum L; when the sign of Lap is −, L of the pixel of interest tendsto assume a value, the absolute value of which is closer to maximum Lthan minimum L. Hence, when the CPU 11 determines a substitute pixelposition with respect to the sign of Lap to substitute the pixelposition, as shown in Table 1 below, image signal values shown in FIG.13A can be adjusted, as shown in FIG. 13C.

If a maximum value is always selected when Lap=0, an edge always becomesthin; if a minimum value is always selected, an edge always becomesbold. To solve such problem, the Lap values of pixels before and afterthe pixel of interest are totaled, and a substitute candidate pixel isdetermined based on the sign of the total.

TABLE 1 Relationship between secondary derivative sign and substitutepixel position Total Lap Sign of Lap Sign of Pixel Previous and NextSubstitute Pixel of Interest Pixels Position + Minimum L − Maximum L 0 +Minimum L 0 − Maximum L 0 0 Maximum L

<Adjusting Level Defining STEP 2513: Define Substitute Level based onAbsolute Value of Variation Acceleration>

The CPU 11 adaptively defines a substitute level C1 in accordance withthe absolute value of the variable acceleration calculated in STEP 2511.FIG. 13C can be obtained by defining C1=1 irrespective of the absolutevalue of the variation acceleration. However, if C1=1 is always defined,jaggy often stands out. Hence, a substation example that can suppressjaggy and can emphasize an edge more than FIG. 13B will be explained.

FIG. 15A is a graph for explaining the C1 defining process in STEP 2513:the abscissa plots the absolute value of the variation acceleration, andthe ordinate plots C1. In case of the variation acceleration which isnear the edge center and is smaller than a ninth threshold, C1 isdefined to be zero so as not to substitute the pixel position. Thereason why the pixel position near the edge center is not substituted isto obscure generation of jaggy. In case of the absolute value of thevariation acceleration which is separated away from the edge center andis larger than a 10th threshold, C1 is defined to be 1 to substitute thepixel position. In case of the absolute value of the variationacceleration falling within the range between the ninth and 10ththresholds (both inclusive), C1, which progressively changes forrespective absolute values of the variation accelerations, is adaptivelydefined, so that C1=0 when the absolute value of the variationacceleration=the ninth threshold and C1=1 when the absolute value of thevariation acceleration=the 10th threshold. As a result, switching of theprocesses can be obscured. More specifically, the CPU 11 can adaptivelydefine C1 with reference to FIG. 15A or using:

C1=(absolute value of variation acceleration−ninth threshold)/(10ththreshold−ninth threshold)  (16)

<Adjusting Level Defining STEP 2514: Define Substitute Level based onVariation Time Count>

The CPU 11 adaptively defines a substitute level Cz in accordance withthe variation time count calculated in STEP 2506. The CPU 11 adaptivelydefines Cz based on characteristics shown in FIG. 15B using 11th and12th thresholds, as in STEP 2507. In case of a bold line area, thevariation time count of which is smaller than the 11th threshold, Cz=1.In case of a thin line or halftone area, the variation time count ofwhich is larger than the 12th threshold, Cz=0. In case of the variationtime count falling within the range between the 11th and 12th thresholds(both inclusive), the CPU 11 can adaptively define Cz using:

Cz=(12th threshold−variation time count)/(12th threshold−11ththreshold)  (17)

<Adjusting Level Defining STEP 2515: Define Substitute Level based onVariation>

The CPU 11 adaptively defines a substitute level Ce in accordance withthe variation calculated in STEP 2505. The CPU 11 adaptively defines Cebased on characteristics shown in FIG. 15C using 13th and 14ththresholds as in STEP 2509. If the variation is smaller than the 13ththreshold, Ce=0. If the variation is larger than the 14th threshold,Ce=1. If the variation falls within the range between the 13th and 14ththresholds (both inclusive), the CPU 11 can adaptively define Ce using:

Ce=(variation−13th threshold)/(14th threshold−13th threshold)  (18)

FIG. 16 is a flowchart of the adjusting process according to the firstembodiment. The adjusting process will be described below alongrespective steps of the flowchart.

<Adjusting Process STEP 2701: Calculate Edge Emphasis Amount>

The CPU 11 calculates differences (edge emphasis amounts) between thepixel value of interest upon applying an edge emphasis filter and thatbefore application for respective colors in the 7×7 RGB areas defined inSTEP 2501. This embodiment will exemplify a case in which a 5×5 edgeemphasis filter is applied to have the pixel of interest as the center.However, the filter size need only be smaller than the process area sizedefined in STEP 2501, and filter coefficients may be appropriatelydefined. FIG. 21A shows an example of filter coefficients of a 5×5 edgeemphasis filter. Let N0 be a pixel value of interest, N1 be a pixelvalue of interest as a result of application of the filter in FIG. 21A,and ΔF be an edge emphasis amount. The CPU 11 can calculate ΔF by:

ΔF=N1−N0  (24)

As shown in FIG. 21B, when the filter coefficient of the pixel ofinterest is defined to be a value obtained by subtracting the filtertotal value in FIG. 21A from the filter coefficient at the position ofthe pixel of interest in FIG. 21A, ΔF can be calculated by only applyingFIG. 21B.

<Adjusting Process STEP 2702: Adjust Edge Emphasis Amount by Fz1>

The CPU 11 adjusts the edge emphasis amount ΔF calculated in STEP 2701using the edge level Fz1 defined in STEP 2507. The CPU 11 calculates anadjusted edge emphasis amount ΔFz1 using:

ΔFz1=Fz1×ΔF  (25)

By the process in STEP 2702, a text area with a small variation timecount can undergo relatively strong edge emphasis, and a halftone areawith a large variation time count can undergo relatively weak edgeemphasis. Hence, the sharpness of text can be enhanced, and moiré can beprevented from being emphasized at the same time.

<Adjusting Process STEP 2703: Adjust Edge Emphasis Amount by Fz2>

The CPU 11 adjusts the edge emphasis amount ΔFz1 calculated in STEP 2702using the edge level Fz2 defined in STEP 2508. The CPU 11 calculates anadjusted edge emphasis amount ΔFz2 using:

ΔFz2=Fz2×ΔFz1  (26)

When Fz2 is defined, as shown in FIG. 12B, the process in STEP 2703 canapply edge emphasis to the bold line area to prevent bordering, and canapply stronger edge emphasis to the thin line area than the bold linearea to enhance the sharpness and increase the density of a blackcharacter.

<Adjusting Process STEP 2704: Adjust Edge Emphasis Amount by Fe>

The CPU 11 adjusts the edge emphasis amount ΔFz2 calculated in STEP 2703using the edge level Fe defined in STEP 2509. The CPU 11 calculates anadjusted edge emphasis amount ΔFe using:

ΔFe=Fe×ΔFz2  (27)

By the process in STEP 2704, the edge area such as a character canundergo relatively strong edge emphasis, and the flat area such as abackground or photo can undergo relatively weak edge emphasis. As aresult, the sharpness of a character can be enhanced, moiré can beprevented from being emphasized, and a photo can be prevented from beingroughened at the same time.

<Adjusting Process STEP 2705: Completion of Edge Emphasis FilterProcess>

The CPU 11 calculates an edge emphasis filter process pixel value Ne byadding the edge emphasis amount ΔFe calculated in STEP 2704 to the pixelvalue N0 of interest, as given by:

Ne=N0+ΔFe  (28)

Note that a process for clipping Ne within a desired range may beinserted.

<Adjusting Process STEP 2706: Calculate Substitute Amount>

The CPU 11 calculates a substitute amount using the pixel value at thesubstitute pixel position determined in STEP 2512. The CPU 11 extractsRGB values at the substitute pixel position determined in STEP 2512 fromthe 7×7 RGB areas defined in STEP 2501. Let N0 be the pixel value ofinterest, C0 be the pixel value at the substitute pixel position, and ΔCbe the substitute amount. Then, the CPU 11 can calculate ΔC using:

ΔC=C0−N0  (19)

<Adjusting Process STEP 2707: Adjust Substitute Amount by C1>

The CPU 11 adjusts the substitute amount ΔC calculated in STEP 2706 bythe substitute level C1 defined in STEP 2513. The CPU 11 calculates anadjusted substitute amount ΔC1 using:

ΔC1=C1×ΔC  (20)

By the process in STEP 2707, the substitution that suppresses generationof jaggy can be applied.

<Adjusting Process STEP 2708: Adjust Substitute Amount by Cz>

The CPU 11 adjusts the substitute amount ΔC1 calculated in STEP 2707 bythe substitute level Cz defined in STEP 2514. The CPU 11 calculates anadjusted substitute amount ΔCz using:

ΔCz=Cz×ΔC1  (21)

By the process in STEP 2708, the substitution that can suppressgeneration of jaggy can be applied by strengthening the substitute levelfor the bold line area, and by weakening that for the thin line area.

<Adjusting Process STEP 2709: Adjust Substitute Amount by Ce>

The CPU 11 adjusts the substitute amount ΔCz calculated in STEP 2708 bythe substitute level Ce defined in STEP 2515. The CPU 11 calculates anadjusted substitute amount ΔCe using:

ΔCe=Ce×ΔCz  (22)

By the process in STEP 2709, an edge area of a character or the like isrelatively strongly substituted to enhance sharpness, and a flat area isrelatively weakly substituted to prevent roughening.

<Adjusting Process STEP 2710: Completion of Substitution Process>

The CPU 11 calculates a pixel value Nc of interest edge-emphasized bymeans of filtering and substitution according to the present inventionby adding the substitute amount ΔCe calculated in STEP 2709 to the edgeemphasis filter-processed value Ne of the pixel of interest, as givenby:

Nc=Ne+ΔCe  (23)

Note that a process for clipping Nc within a desired range may beinserted.

<Effects of Embodiment>

According to this embodiment, since emphasis is applied usingsubstitution, emphasis free from bordering can be executed whileassuring a sharp edge.

Since the edge direction is determined based on the primary derivativeof representative values, a substitute candidate pixel can always beselected from a correct edge direction, and edge emphasis can beeffectively applied. Since the substitute pixel position can bedetermined using the sign of the primary derivative, a pixel with anappropriate luminance value can be selected according to surroundingluminance values L, and edge emphasis can be effectively applied. Whenthe secondary derivative is “0”, since the substitute candidate pixelposition is determined with reference to the secondary derivative valuesof pixels before and after the pixel of interest, edge emphasis can beeffectively applied. Since the edge direction is determined from fourdirections, pixels other than those in the four directions need not bereferred to, thus simplifying the processes. This effect becomes moreconspicuous with increasing reference area size.

As shown in FIG. 20, in case of a 7×7 reference area, 25 out of 49pixels are included in the four directions. In case of a 9×9 referencearea, 33 out of 81 pixels are included in the four directions. Also, incase of an 11×11 reference area, 41 out of 121 pixels are included inthe four directions. That is, in case of a (2n+1)×(2n+1) reference area,(8n+1) out of (2n+1)×(2n+1) pixels are included in the four directions.Hence, the effect of process simplification owing to needlessness ofreference to pixels other than those in the four directions is enhancedmore with increasing n. Upon scanning using the same optical system, theedge blurred amount often increases with increasing resolution. Hence,in such case, the reference area size must be increased. Therefore, thiseffect is larger with increasing resolution. Since the edge direction isdetermined, and the variation acceleration in only the edge direction isthen calculated, variation accelerations other than that in the edgedirection need not be calculated, resulting in simple processes.

Also, the following effect can be provided. FIG. 17 expresses an imagewith gradation from blue on the left side to yellow on the right side.Each box represents a pixel, and three numerical values in that boxrepresent R, G, and B signal values. For example, “0, 0, 240” means R=0,G=0, and B=240, and indicates a blue pixel. An intermediate part fromblue on the left side to yellow on the right side has intermediatecolors from blue to yellow, and illustrates a blurred state due tooptical scanning.

FIG. 18 expresses an image obtained by applying this embodiment to theimage shown in FIG. 17 while the adjusting levels are always set to beCl=Cz=Ce=1. The left half is blue, the right half is yellow, and thereis no intermediate color part. As can be seen from FIG. 18, blurs causedby optical scanning can be adjusted, thus enhancing the sharpness.

FIG. 19 expresses an image obtained by independently executingdetermination and substitution of the substitute pixel positions for R,G, and B signals while the adjusting levels are always set to beCl=Cz=Ce=1 and no L signal is generated. The left side is blue, and theright side is yellow, but R=0, G=0, and B=0, i.e., a black part isformed in their intermediate part. This is a problem posed when thesubstitute pixel positions are independently determined for R, G, and B.This problem can be solved by calculating luminance L and determining acommon substitute pixel position based on L like in this embodiment.

As described above, in this embodiment, luminance L is generated fromRGB signals in STEP 2501, and the substitute pixel position isdetermined based on luminance L in STEP 2512. With this process, onlyone substitute pixel position common to RGB can always be determinedindependently of the signal distribution patterns of RGB signal values.This allows adjusting free from any problem of color misregistrationcaused by determining independent substitute pixel positions for RGBsignals.

As can be seen from the above description, when a G signal is used as anL signal without using the L signal of this embodiment, only onesubstitute pixel position common to RGB can similarly be determined.Hence, this also allows adjusting free from any problem of colormisregistration.

In this embodiment, since the substitute level for the substitutecandidate pixel is continuously changed with respect to a color image,the tonality of a natural image can be maintained. Also, by increasing atext emphasis amount and decreasing a halftone emphasis amount, a morefavorable result free from any moiré can be obtained.

Upon independently detecting edge directions without using anyrepresentative value signals, different edge directions may be detectedfor RGB. In this case, different substitute pixel positions may beselected for RGB, and Grd and Lap values must also be calculated forRGB. Using this embodiment, Grd and Lap values need only calculated foronce based on representative value signals, and processes can be simplecompared to independent calculations for RGB.

As described above, since the edge direction is detected fromrepresentative value signals in this embodiment, the edge in the samedirection can always be detected. This can also reduce the calculationvolume, resulting in simple processes.

OTHER EMBODIMENTS

The embodiments of the present invention have been explained in detail.The present invention may be applied to either a system constituted by aplurality of devices, or an apparatus consisting of a single device.

Note that the present invention includes a case wherein the invention isachieved by directly or remotely supplying a program that implements thefunctions of the aforementioned embodiments to a system or apparatus,and reading out and executing the supplied program code by a computer ofthat system or apparatus. Therefore, the technical scope of the presentinvention includes the program code itself installed in a computer toimplement the functional processes of the present invention using thecomputer.

In this case, the form of program is not particularly limited, and anobject code, a program to be executed by an interpreter, script data tobe supplied to an OS, and the like may be used as long as they have theprogram function.

As a recording medium for supplying the program, for example, a floppy®disk, hard disk, optical disk, and magneto-optical disk may be used.Also, MO, CD-ROM, CD-R, CD-RW, magnetic tape, nonvolatile memory card,ROM, DVD (DVD-ROM, DVD-R), and the like may be used.

As another use method, a connection is established to the Internet siteusing a browser of a client PC, and the program itself according to thepresent invention or a file that further includes an automaticinstallation function may be downloaded to a recording medium such as ahard disk or the like. Also, the program code that forms the program ofthe present invention may be segmented into a plurality of files, whichmay be downloaded from different homepages. That is, the presentinvention includes a WWW server which makes a plurality of usersdownload a program required to implement the functional processes of thepresent invention by the computer. Furthermore, a storage medium such asa CD-ROM or the like, which stores the encrypted program of the presentinvention, may be delivered to the users. The user who has cleared apredetermined condition may be allowed to download key information thatdecrypts the program from a homepage via the Internet, and the encryptedprogram may be executed using that key information to be installed on acomputer, thus implementing the present invention.

Moreover, the functions of the aforementioned embodiments can beimplemented by some or all of actual processes executed by an OS or thelike which runs on a computer based on instructions of the program.

In addition, the scope of the present invention includes a case in whichthe program according to the present invention is written in a memory ofa function expansion unit of a PC, and a CPU equipped on that functionexpansion unit executes some or all of actual processes.

According to the present invention, an appropriate adjusting processingcan be applied to an image including multicolor image signals.

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modifications and equivalent structures andfunctions.

This application claims the benefit of Japanese Patent Application No.2006-180379 filed on Jun. 29, 2006, which is hereby incorporated byreference herein in its entirety.

1. An image processing apparatus for applying an adjusting process to animage including multicolor image signals, comprising: an extracting unitarranged to extract an image area including a pixel to be processed; ageneration unit arranged to generate one representative signal valuefrom signal values corresponding to a plurality of colors of pixelsincluded in the image area; a feature amount calculation unit arrangedto calculate a feature amount associated with the pixel to be processedbased on the representative signal value; a unit arranged to select asubstitute candidate pixel common to the plurality of colors from theimage area based on the calculated feature amount; and a unit arrangedto substitute signal values of the pixel to be processed by new pixelvalues, which are calculated from the signal values of the pixel to beprocessed and signal values of the substitute candidate pixel.
 2. Theapparatus according to claim 1, wherein said feature amount calculationunit calculates, as the feature amount, a value obtained by calculatinga primary derivative of the representative signal value and a valueobtained by calculating a secondary derivative of the representativesignal value.
 3. The apparatus according to claim 1, wherein themulticolor image signals are RGB signals.
 4. The apparatus according toclaim 1, wherein the multicolor image signals are signals includinglightness and color difference components.
 5. The apparatus according toclaim 1, wherein said generation unit generates the representativesignal value from a signal value representing a luminance.
 6. Theapparatus according to claim 1, wherein said generation unit generatesthe representative signal value from one of the multicolor imagesignals.
 7. An image processing apparatus for applying an adjustingprocess to an image including multicolor image signals, comprising: anextracting unit arranged to extract an image area including a pixel tobe processed; an edge direction determination unit arranged to determinean edge direction in the image area; a selection unit arranged to selecta substitute candidate pixel from a pixel group included in the edgedirection; and a unit arranged to substitute signal values of the pixelto be processed by new pixel values, which are calculated from thesignal values of the pixel to be processed and signal values of thesubstitute candidate pixel.
 8. The apparatus according to claim 7,further comprising a generation unit arranged to generate onerepresentative signal value from signal values corresponding to aplurality of colors of pixels included in the image area, wherein saidedge direction determination unit determines the edge direction using avalue obtained by calculating a primary derivative of the representativesignal value.
 9. The apparatus according to claim 7, wherein when asecondary derivative value obtained by calculating a secondaryderivative of the signal values of the pixel to be processed ispositive, said selection unit selects a pixel with minimum signal valuesas the substitute candidate pixel, and when the secondary derivativevalue is negative, said selection unit selects a pixel with maximumsignal values as the substitute candidate pixel.
 10. The apparatusaccording to claim 9, wherein when the secondary derivative value is“0”, said selection unit selects a pixel with minimum or maximum signalvalues with reference to a sign of a value obtained by calculating asecondary derivative of signal values of pixels which neighbor the pixelto be processed.
 11. The apparatus according to claim 7, wherein saidedge direction determination unit compares primary derivative values ofsignal values of pixel sequences in a plurality of directions includingthe pixel to be processed in the image area, and determines a directionthat exhibits a maximum primary derivative value as the edge direction.12. An image processing apparatus for applying an adjusting process toan image including multicolor image signals, comprising: an extractingunit arranged to extract an image area including a pixel to beprocessed; a generation unit arranged to generate one representativesignal value from signal values corresponding to a plurality of colorsof pixels included in the image area; a feature amount calculation unitarranged to calculate a feature amount associated with the pixel to beprocessed based on the representative signal value; a unit arranged toselect a substitute candidate pixel common to the plurality of colorsfrom the image area based on the calculated feature amount; an edgedirection determination unit arranged to determine an edge directionbased on the representative signal value in the image area; a selectionunit arranged to select a substitute candidate pixel from a pixel groupincluded in the edge direction; and a unit arranged to substitute signalvalues of the pixel to be processed by new pixel values, which arecalculated from the signal values of the pixel to be processed andsignal values of the substitute candidate pixel.
 13. The apparatusaccording to claim 7, further comprising a generation unit arranged togenerate one representative signal value from signal valuescorresponding to a plurality of colors of pixels included in the imagearea, wherein said edge direction determination unit determines the edgedirection using a value obtained by calculating a primary derivative ofthe representative signal value.
 14. The apparatus according to claim 7,wherein when a secondary derivative value obtained by calculating asecondary derivative of the signal values of the pixel to be processedis positive, said selection unit selects a pixel with minimum signalvalues as the substitute candidate pixel, and when the secondaryderivative value is negative, said selection unit selects a pixel withmaximum signal values as the substitute candidate pixel.
 15. Theapparatus according to claim 10, wherein when the secondary derivativevalue is “0”, said selection unit selects a pixel with minimum ormaximum signal values with reference to a sign of a value obtained bycalculating a secondary derivative of signal values of pixels whichneighbor the pixel to be processed.
 16. The apparatus according to claim7, wherein said edge direction determination unit compares primaryderivative values of signal values of pixel sequences in a plurality ofdirections including the pixel to be processed in the image area, anddetermines a direction that exhibits a maximum primary derivative valueas the edge direction.
 17. An image processing method for applying anadjusting process to an image including multicolor image signals,comprising the steps of: extracting an image area including a pixel tobe processed; generating one representative signal value from signalvalues corresponding to a plurality of colors of pixels included in theimage area; calculating a feature amount associated with the pixel to beprocessed based on the representative signal value; selecting asubstitute candidate pixel common to the plurality of colors from theimage area based on the calculated feature amount; and substitutingsignal values of the pixel to be processed by new pixel values, whichare calculated from the signal values of the pixel to be processed andsignal values of the substitute candidate pixel.
 18. An image processingmethod for applying an adjusting process to an image includingmulticolor image signals, comprising the steps of: extracting an imagearea including a pixel to be processed; generating one representativesignal value from signal values corresponding to a plurality of colorsof pixels included in the image area; calculating a feature amountassociated with the pixel to be processed based on the representativesignal value; selecting a substitute candidate pixel common to theplurality of colors from the image area based on the calculated featureamount; determining an edge direction based on the representative signalvalue in the image area; selecting a substitute candidate pixel from apixel group included in the edge direction; and substituting signalvalues of the pixel to be processed by new pixel values, which arecalculated from the signal values of the pixel to be processed andsignal values of the substitute candidate pixel.
 19. An image processingprogram for applying an adjusting process to an image includingmulticolor image signals, said program making a computer execute thesteps of: extracting an image area including a pixel to be processed;generating one representative signal value from signal valuescorresponding to a plurality of colors of pixels included in the imagearea; calculating a feature amount associated with the pixel to beprocessed based on the representative signal value; selecting asubstitute candidate pixel common to the plurality of colors from theimage area based on the calculated feature amount; and substitutingsignal values of the pixel to be processed by new pixel values, whichare calculated from the signal values of the pixel to be processed andsignal values of the substitute candidate pixel.
 20. An image processingprogram for applying an adjusting process to an image includingmulticolor image signals, said program making a computer execute thesteps of: extracting an image area including a pixel to be processed;generating one representative signal value from signal valuescorresponding to a plurality of colors of pixels included in the imagearea; calculating a feature amount associated with the pixel to beprocessed based on the representative signal value; selecting asubstitute candidate pixel common to the plurality of colors from theimage area based on the calculated feature amount; determining an edgedirection based on the representative signal value in the image area;selecting a substitute candidate pixel from a pixel group included inthe edge direction; and substituting signal values of the pixel to beprocessed by new pixel values, which are calculated from the signalvalues of the pixel to be processed and signal values of the substitutecandidate pixel.